Skip to content

perf: store the build rows of repeated hash join keys together - #27530

Open
andygrove wants to merge 5 commits into
apache:mainfrom
andygrove:hash-join-grouped-rows
Open

andygrove wants to merge 5 commits into
apache:mainfrom
andygrove:hash-join-grouped-rows

Conversation

@andygrove

@andygrove andygrove commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Note

This is the first half of #26447, split out as suggested in that PR's review. It covers the hash maps (JoinHashMapU32, JoinHashMapU64). #26447 now holds the ArrayMap half on top of this PR.

Which issue does this PR close?

Rationale for this change

When a build-side key repeats, HashJoinExec keeps the build rows of each key in a chain. The map holds the first row and next[row] holds the one after it. A probe row follows the chain with one load for every match, and each load depends on the one before, so nothing overlaps them. Once the build side no longer fits in the cache, most of those loads are misses. This is what makes joins with many build rows per key, such as the join of catalog_sales and inventory in TPC-DS q72, several times slower than a sort-merge join.

What changes are included in this PR?

JoinHashMapU32 and JoinHashMapU64 store the build rows of each hash value next to each other, so that a probe row reads its matches in order:

  • After the last build batch is inserted, the rows are grouped when the hash values have at least two rows each on average. The map then holds the position of each hash value's group, which starts with its number of rows, and the chain is freed. Rows left out of the map, such as those with NULL keys, count towards neither the threshold nor the groups.
  • The groups take one more index per distinct hash value than the chain. The build reserves that memory first, and keeps the chain if it can't. Grouping happens after every other reservation the build needs, so it never makes the build fail.
  • Grouping follows 16 chains at once, so that their loads overlap. It holds up to 4096 rows of each chain until the chain ends, and measures longer chains to place them in a second pass. So it holds at most 16 x 4096 indices besides the groups.
  • The probe copies a probe row's matches with one slice copy. ProbeOffset::MidChain::next is then the position in the groups to resume from.
  • A new grouped_map_count metric counts the maps that were grouped.

Matches come out in the same order as before, so the join's output does not change. ArrayMap keeps its chain in this PR.

One hot key among unique keys keeps its chain, because the keys have fewer than two rows each on average. #27614 tracks grouping only the repeated keys, so that this case is grouped too.

Benchmarks

datafusion-cli (release-nonlto) on an AMD 7950X3D (16 cores), 16 partitions. Each figure is the mean of the warm runs of the query, alternating between the builds. All queries use sparse keys, so the build side is a JoinHashMap. The first two cases build 64M rows over 640K keys, about 100 rows per key, as in the reproduction from #17494. The others build 64M rows and probe 96M:

Case main this PR Grouped
Repeated keys, 96M probe rows (the #17494 reproduction) 77.7 s 33.6 s yes
Repeated keys, 16K probe rows 0.49 s 0.59 s yes
One hot key (1% of 64M build rows) among unique keys 1.85 s 1.80 s no
Unique keys, 64M build rows 1.76 s 1.73 s no

With 16K probe rows there is too little to probe to pay for grouping 64M build rows. The hot key stays chained, see #27614. Every query returns the same result as on main.

I'll run hj with the benchmark bot, now that #27528 has merged and Q28 has a baseline on main.

What is the testing strategy for this PR?

  • New unit tests in join_hash_map.rs compare grouped maps with the chain or with a brute-force result. They cover rows inserted in batches from the last to the first, a row left out as for a NULL key, limits from 1 to 8192, NULL probe keys, chains long enough for the second pass, the size of the groups, the memory freed, and short chains that stay ungrouped.
  • grouping_never_fails_the_build in hash_join/exec.rs checks that a build that fits without its groups never fails for them, and groups its rows once the memory for them is available.
  • The .slt changes only add the new metric to EXPLAIN ANALYZE output.

Are there any user-facing changes?

There's a new grouped_map_count metric, documented in metrics.md. JoinHashMapType, which is public for internal use only, gains group_rows with a default implementation.

This PR was prepared with AI assistance (Claude Code) and reviewed by me.

🤖 Generated with Claude Code

andygrove and others added 2 commits November 17, 2026 07:49
When a build-side key repeats, a probe row follows the key's chain of
build rows with one dependent load for every match, which misses the
cache when the build side is large. After the build, if the hash values
have at least two rows each on average and the memory pool can hold
the groups, store each hash value's rows next to each other and drop
the chain, so a probe row reads its matches from one place.

Grouping happens after every reservation the build needs and is
skipped without the memory for it, so it never fails a build. This
change covers the hash maps. `ArrayMap` keeps its chain for now.

Adds the `grouped_map_count` metric.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@andygrove

Copy link
Copy Markdown
Member Author

Split out of #26447. Earlier review discussion is there.

@github-actions github-actions Bot added documentation Improvements or additions to documentation sqllogictest SQL Logic Tests (.slt) physical-plan Changes to the physical-plan crate labels Nov 17, 2026

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andygrove. The split is much easier to follow, and the earlier points (reservation order, NULL rows, the KeyRows state, the dead get_matched_indices branch) all landed.

Design questions:

  • Concurrent builds. "Never fails the build" holds for the build that groups. In Partitioned mode every partition builds at once from the same pool, and grouping holds the chain and the groups together (about twice the index memory at the peak), then keeps one more index per distinct hash value for the whole probe. A sibling partition's mandatory try_grow, such as in concat_build_batches, can then fail where it fits on main. grouping_never_fails_the_build only covers one partition. Is this worth a sentence in the group_rows doc, or some headroom before grouping?
  • Skew in CollectLeft. Could the follow-up for one hot key among unique keys get an issue linked from the PR body? One direction: every group pays 1 + its row count, including hash values with a single row, which is what forces the average threshold. If single-row hash values kept their row in the map and only repeated ones pointed at a group, the cost would follow the repeated rows. The threshold could then count those instead of the average. The catch is a tag bit in the map value.

Nit: group_rows_range is pub(crate) but only used in join_hash_map.rs until the ArrayMap half lands.

let to_skip = match offset {
AtRow { row } => row,
// Resume at position `next` of the matches of probe row `row`
MidChain { row, next } => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MidChain branch repeats the loop body for a single row. If the resume position goes into the loop instead, there is one call site and the expect goes away. emit_grouped_rows could then be inlined into the loop:

let (first_row, mut resume_pos) = match offset {
    AtRow { row } => (row, None),
    MidChain { row, next } => (row, Some(next as usize)),
};
for row in first_row..num_rows {
    let pos = resume_pos.take();
    let Some((start, end)) = rows_of(row) else {
        continue;
    };
    if let Some(next_offset) = emit_grouped_rows(
        &rows[..end],
        pos.unwrap_or(start),
        row,
        &mut remaining,
        input_indices,
        match_indices,
        row == num_rows - 1,
    ) {
        return Some(next_offset);
    }
}
None

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in dd8ecf4. The resume position goes into the loop, and the emit helper is inlined.

match_indices,
)
}
KeyRows::Grouped { groups } => grouped_matched_indices(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This arm, and the Chain arm above it, are identical in the JoinHashMapU64 impl. A generic free function that takes &HashTable<(u64, T)> and &KeyRows<T> and does the match itself, next to get_matched_indices_with_limit_offset, would turn both impls into one call and keep the rows_of closure in one place.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in dd8ecf4. Both impls call key_rows_matched_indices.

// inserted before it, while the rows left out of the map, such as NULL
// keys, keep `0`.
let zero = T::usize_as(0);
let chained_rows = next.iter().filter(|&&index| index != zero).count();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes a full pass over next for every build with a repeated key, including the ones that stay chained. On main that costs nothing, and here it runs serially in CollectLeft (I haven't measured it). update_from_iter already sees each chained insert. If it returned the number of chained rows instead of bool (the symmetric hash join ignores the result), KeyRows::Chain could hold chained_rows: usize. Then !chained becomes chained_rows == 0 and this pass goes away.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in dd8ecf4. update_from_iter returns the number of chained rows, and KeyRows::Chain keeps chained_rows. The pass over next is gone.

map: &mut HashTable<(u64, T)>,
rows: &mut KeyRows<T>,
reservation: &MemoryReservation,
mem_used: &Gauge,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ArrayMap::try_new leaves build_mem_used to the caller (metrics.build_mem_used.add(array_map.size())). Following that here would keep metrics out of join_hash_map.rs. group_rows would take only the reservation, and build_in_memory would update the gauge from map_reservation.size() before and after. That drops the Gauge import, the unused _mem_used in the default method, the gauge half of the reserved test helper, and the mem_used assertions that repeat reservation.size().

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in dd8ecf4. group_rows takes only the reservation, and build_in_memory updates build_mem_used from the change in map_reservation.size().


/// An unbounded pool that remembers the most it had reserved at once
#[derive(Debug, Default)]
struct PeakPool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PeakRecordingPool in datafusion_execution::memory_pool already records the high-water mark. PeakRecordingPool::new(Arc::new(UnboundedMemoryPool::default())) with peak_reserved() replaces this pool and its Display impl. The aggregate tests in aggregates/mod.rs use it the same way.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in dd8ecf4. Switched to PeakRecordingPool.

- Count chained rows on insert instead of scanning next before grouping
- Share the probe of both hash map impls in one generic function
- Resume a grouped probe in the main loop
- Update build_mem_used in build_in_memory, not in group_rows
- Document the effect of grouping on builds sharing the pool
- Use PeakRecordingPool in grouping_never_fails_the_build
- Make group_rows_range private

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@andygrove

Copy link
Copy Markdown
Member Author

Thanks @comphead. All inline points are addressed in dd8ecf4, along with the group_rows_range visibility nit.

Concurrent builds: I added a paragraph to the group_rows doc. It says that while grouping, the reservation holds both the chain and the groups. Afterwards it holds one more index per distinct key, so sibling partitions sharing the pool can find less memory than on main. I'd rather not guess a headroom margin without a case that shows the failure. Happy to add one if you have a figure in mind.

Skew in CollectLeft: agreed, this belongs in a follow-up. I'll open an issue for the hot-key case, with your idea of keeping single-row keys in the map behind a tag bit, and link it from the PR body. The hj hot-key query from #27528 gives it a baseline.

# Conflicts:
#	datafusion/physical-plan/src/joins/hash_join/exec.rs
@Dandandan

Copy link
Copy Markdown
Contributor

run benchmarks

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c7041665205-5525-nlxc6 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing hash-join-grouped-rows (104d556) to 477c7b9 (merge-base) diff

Run configuration
run benchmark tpcds

Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c7041665205-5524-8dlbt 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing hash-join-grouped-rows (104d556) to 477c7b9 (merge-base) diff

Run configuration
run benchmark clickbench_partitioned

Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c7041665205-5526-gllgc 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing hash-join-grouped-rows (104d556) to 477c7b9 (merge-base) diff

Run configuration
run benchmark tpch

Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing hash-join-grouped-rows (104d556) to 477c7b9 (merge-base) diff

Run configuration
run benchmark tpch
CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and hash-join-grouped-rows
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query     ┃     HEAD ┃ hash-join-grouped-rows ┃    Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 1  │ 40.60 ms │               40.30 ms │ no change │
│ QQuery 2  │ 18.33 ms │               18.35 ms │ no change │
│ QQuery 3  │ 28.35 ms │               28.27 ms │ no change │
│ QQuery 4  │ 17.04 ms │               17.25 ms │ no change │
│ QQuery 5  │ 35.27 ms │               35.15 ms │ no change │
│ QQuery 6  │ 13.56 ms │               13.66 ms │ no change │
│ QQuery 7  │ 41.45 ms │               40.09 ms │ no change │
│ QQuery 8  │ 39.91 ms │               39.98 ms │ no change │
│ QQuery 9  │ 48.48 ms │               48.56 ms │ no change │
│ QQuery 10 │ 39.96 ms │               39.85 ms │ no change │
│ QQuery 11 │ 13.26 ms │               13.15 ms │ no change │
│ QQuery 12 │ 17.31 ms │               17.48 ms │ no change │
│ QQuery 13 │ 38.91 ms │               38.97 ms │ no change │
│ QQuery 14 │ 24.00 ms │               24.30 ms │ no change │
│ QQuery 15 │ 30.93 ms │               30.98 ms │ no change │
│ QQuery 16 │ 13.31 ms │               13.28 ms │ no change │
│ QQuery 17 │ 66.81 ms │               67.18 ms │ no change │
│ QQuery 18 │ 39.82 ms │               40.53 ms │ no change │
│ QQuery 19 │ 31.76 ms │               32.72 ms │ no change │
│ QQuery 20 │ 31.70 ms │               32.07 ms │ no change │
│ QQuery 21 │ 55.15 ms │               54.34 ms │ no change │
│ QQuery 22 │ 13.18 ms │               13.30 ms │ no change │
└───────────┴──────────┴────────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary                     ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)                     │ 699.07ms │
│ Total Time (hash-join-grouped-rows)   │ 699.75ms │
│ Average Time (HEAD)                   │  31.78ms │
│ Average Time (hash-join-grouped-rows) │  31.81ms │
│ Queries Faster                        │        0 │
│ Queries Slower                        │        0 │
│ Queries with No Change                │       22 │
│ Queries with Failure                  │        0 │
└───────────────────────────────────────┴──────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and hash-join-grouped-rows
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query     ┃                           HEAD ┃         hash-join-grouped-rows ┃       Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1  │ 40.60 / 42.65 ±1.21 / 43.83 ms │ 40.30 / 41.66 ±1.50 / 43.82 ms │    no change │
│ QQuery 2  │ 18.33 / 18.55 ±0.33 / 19.20 ms │ 18.35 / 18.61 ±0.28 / 19.12 ms │    no change │
│ QQuery 3  │ 28.35 / 29.45 ±2.05 / 33.54 ms │ 28.27 / 28.71 ±0.56 / 29.79 ms │    no change │
│ QQuery 4  │ 17.04 / 17.28 ±0.13 / 17.39 ms │ 17.25 / 17.48 ±0.24 / 17.94 ms │    no change │
│ QQuery 5  │ 35.27 / 35.82 ±0.95 / 37.70 ms │ 35.15 / 35.86 ±0.70 / 37.12 ms │    no change │
│ QQuery 6  │ 13.56 / 13.99 ±0.73 / 15.46 ms │ 13.66 / 14.16 ±0.76 / 15.67 ms │    no change │
│ QQuery 7  │ 41.45 / 42.57 ±0.74 / 43.35 ms │ 40.09 / 41.28 ±0.79 / 42.46 ms │    no change │
│ QQuery 8  │ 39.91 / 40.33 ±0.47 / 41.20 ms │ 39.98 / 41.06 ±1.73 / 44.51 ms │    no change │
│ QQuery 9  │ 48.48 / 49.49 ±0.79 / 50.65 ms │ 48.56 / 49.33 ±0.56 / 50.18 ms │    no change │
│ QQuery 10 │ 39.96 / 40.18 ±0.31 / 40.79 ms │ 39.85 / 40.39 ±0.29 / 40.71 ms │    no change │
│ QQuery 11 │ 13.26 / 13.41 ±0.23 / 13.87 ms │ 13.15 / 13.35 ±0.21 / 13.75 ms │    no change │
│ QQuery 12 │ 17.31 / 17.75 ±0.30 / 18.01 ms │ 17.48 / 18.02 ±0.47 / 18.90 ms │    no change │
│ QQuery 13 │ 38.91 / 39.87 ±1.72 / 43.31 ms │ 38.97 / 39.19 ±0.22 / 39.57 ms │    no change │
│ QQuery 14 │ 24.00 / 24.33 ±0.29 / 24.86 ms │ 24.30 / 25.55 ±2.21 / 29.97 ms │ 1.05x slower │
│ QQuery 15 │ 30.93 / 32.01 ±1.11 / 33.98 ms │ 30.98 / 31.81 ±0.73 / 32.97 ms │    no change │
│ QQuery 16 │ 13.31 / 13.51 ±0.11 / 13.63 ms │ 13.28 / 13.41 ±0.07 / 13.49 ms │    no change │
│ QQuery 17 │ 66.81 / 67.64 ±0.73 / 68.89 ms │ 67.18 / 67.77 ±0.83 / 69.42 ms │    no change │
│ QQuery 18 │ 39.82 / 41.08 ±0.80 / 42.23 ms │ 40.53 / 41.00 ±0.54 / 42.00 ms │    no change │
│ QQuery 19 │ 31.76 / 32.69 ±1.58 / 35.83 ms │ 32.72 / 35.27 ±3.02 / 41.02 ms │ 1.08x slower │
│ QQuery 20 │ 31.70 / 32.12 ±0.40 / 32.80 ms │ 32.07 / 32.25 ±0.12 / 32.44 ms │    no change │
│ QQuery 21 │ 55.15 / 56.60 ±2.07 / 60.67 ms │ 54.34 / 54.70 ±0.28 / 55.20 ms │    no change │
│ QQuery 22 │ 13.18 / 13.46 ±0.23 / 13.87 ms │ 13.30 / 13.57 ±0.16 / 13.80 ms │    no change │
└───────────┴────────────────────────────────┴────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary                     ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)                     │ 714.81ms │
│ Total Time (hash-join-grouped-rows)   │ 714.42ms │
│ Average Time (HEAD)                   │  32.49ms │
│ Average Time (hash-join-grouped-rows) │  32.47ms │
│ Queries Faster                        │        0 │
│ Queries Slower                        │        2 │
│ Queries with No Change                │       20 │
│ Queries with Failure                  │        0 │
└───────────────────────────────────────┴──────────┘

Resource Usage

tpch — base (merge-base)

Metric Value
Wall time 5.0s
Peak memory 1.2 GiB
Avg memory 495.3 MiB
CPU user 19.1s
CPU sys 1.6s
Peak spill 0 B

tpch — branch

Metric Value
Wall time 5.0s
Peak memory 1.2 GiB
Avg memory 488.9 MiB
CPU user 19.0s
CPU sys 1.7s
Peak spill 0 B

File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing hash-join-grouped-rows (104d556) to 477c7b9 (merge-base) diff

Run configuration
run benchmark tpcds
CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and hash-join-grouped-rows
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ hash-join-grouped-rows ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │    6.04 ms │                5.98 ms │     no change │
│ QQuery 2  │   81.55 ms │               80.99 ms │     no change │
│ QQuery 3  │   30.13 ms │               29.68 ms │     no change │
│ QQuery 4  │  499.97 ms │              498.11 ms │     no change │
│ QQuery 5  │   52.92 ms │               51.59 ms │     no change │
│ QQuery 6  │   37.26 ms │               38.45 ms │     no change │
│ QQuery 7  │   73.56 ms │               73.89 ms │     no change │
│ QQuery 8  │   38.33 ms │               38.06 ms │     no change │
│ QQuery 9  │   54.54 ms │               52.79 ms │     no change │
│ QQuery 10 │   62.12 ms │               61.80 ms │     no change │
│ QQuery 11 │  321.42 ms │              315.99 ms │     no change │
│ QQuery 12 │   30.00 ms │               29.64 ms │     no change │
│ QQuery 13 │  118.25 ms │              115.64 ms │     no change │
│ QQuery 14 │  386.16 ms │              386.65 ms │     no change │
│ QQuery 15 │   62.74 ms │               62.21 ms │     no change │
│ QQuery 16 │    6.58 ms │                6.76 ms │     no change │
│ QQuery 17 │   76.94 ms │               78.15 ms │     no change │
│ QQuery 18 │   97.95 ms │               98.58 ms │     no change │
│ QQuery 19 │   43.22 ms │               42.50 ms │     no change │
│ QQuery 20 │   36.72 ms │               38.90 ms │  1.06x slower │
│ QQuery 21 │   17.56 ms │               18.10 ms │     no change │
│ QQuery 22 │   66.26 ms │               70.96 ms │  1.07x slower │
│ QQuery 23 │  323.75 ms │              327.67 ms │     no change │
│ QQuery 24 │  210.53 ms │              212.49 ms │     no change │
│ QQuery 25 │  108.87 ms │              107.87 ms │     no change │
│ QQuery 26 │   47.21 ms │               47.05 ms │     no change │
│ QQuery 27 │    5.65 ms │                5.73 ms │     no change │
│ QQuery 28 │   59.92 ms │               60.15 ms │     no change │
│ QQuery 29 │   92.64 ms │               94.10 ms │     no change │
│ QQuery 30 │   34.65 ms │               34.49 ms │     no change │
│ QQuery 31 │  109.36 ms │              112.01 ms │     no change │
│ QQuery 32 │   20.96 ms │               21.44 ms │     no change │
│ QQuery 33 │   39.08 ms │               38.90 ms │     no change │
│ QQuery 34 │    6.22 ms │                6.39 ms │     no change │
│ QQuery 35 │   75.60 ms │               77.60 ms │     no change │
│ QQuery 36 │    6.24 ms │                6.30 ms │     no change │
│ QQuery 37 │    7.65 ms │                7.57 ms │     no change │
│ QQuery 38 │   64.77 ms │               64.09 ms │     no change │
│ QQuery 39 │   77.06 ms │               79.21 ms │     no change │
│ QQuery 40 │   24.59 ms │               25.57 ms │     no change │
│ QQuery 41 │   13.76 ms │               13.60 ms │     no change │
│ QQuery 42 │   24.43 ms │               24.84 ms │     no change │
│ QQuery 43 │    5.30 ms │                5.15 ms │     no change │
│ QQuery 44 │    6.51 ms │                6.09 ms │ +1.07x faster │
│ QQuery 45 │   46.62 ms │               44.45 ms │     no change │
│ QQuery 46 │   13.35 ms │               12.71 ms │     no change │
│ QQuery 47 │  259.41 ms │              253.39 ms │     no change │
│ QQuery 48 │   93.90 ms │               93.95 ms │     no change │
│ QQuery 49 │   72.95 ms │               73.12 ms │     no change │
│ QQuery 50 │   58.60 ms │               58.69 ms │     no change │
│ QQuery 51 │   97.21 ms │               98.53 ms │     no change │
│ QQuery 52 │   25.32 ms │               25.36 ms │     no change │
│ QQuery 53 │   30.70 ms │               30.53 ms │     no change │
│ QQuery 54 │   55.10 ms │               56.11 ms │     no change │
│ QQuery 55 │   24.78 ms │               24.75 ms │     no change │
│ QQuery 56 │   39.87 ms │               39.80 ms │     no change │
│ QQuery 57 │  179.77 ms │              178.81 ms │     no change │
│ QQuery 58 │  113.57 ms │              113.11 ms │     no change │
│ QQuery 59 │  118.82 ms │              118.60 ms │     no change │
│ QQuery 60 │   39.70 ms │               39.53 ms │     no change │
│ QQuery 61 │    9.53 ms │                9.63 ms │     no change │
│ QQuery 62 │   44.75 ms │               45.91 ms │     no change │
│ QQuery 63 │   30.53 ms │               30.31 ms │     no change │
│ QQuery 64 │  360.19 ms │              373.00 ms │     no change │
│ QQuery 65 │   72.37 ms │               72.85 ms │     no change │
│ QQuery 66 │   75.70 ms │               77.04 ms │     no change │
│ QQuery 67 │  242.10 ms │              243.80 ms │     no change │
│ QQuery 68 │    7.08 ms │                6.95 ms │     no change │
│ QQuery 69 │   56.99 ms │               57.15 ms │     no change │
│ QQuery 70 │  105.67 ms │              108.75 ms │     no change │
│ QQuery 71 │   35.56 ms │               36.00 ms │     no change │
│ QQuery 72 │ 1820.40 ms │             1765.25 ms │     no change │
│ QQuery 73 │    5.57 ms │                5.93 ms │  1.06x slower │
│ QQuery 74 │  224.56 ms │              224.12 ms │     no change │
│ QQuery 75 │  141.47 ms │              143.77 ms │     no change │
│ QQuery 76 │   35.80 ms │               35.41 ms │     no change │
│ QQuery 77 │   61.48 ms │               62.07 ms │     no change │
│ QQuery 78 │  285.27 ms │              286.03 ms │     no change │
│ QQuery 79 │   69.47 ms │               70.56 ms │     no change │
│ QQuery 80 │   99.88 ms │               99.17 ms │     no change │
│ QQuery 81 │   27.15 ms │               27.79 ms │     no change │
│ QQuery 82 │   17.26 ms │               17.48 ms │     no change │
│ QQuery 83 │   34.12 ms │               34.50 ms │     no change │
│ QQuery 84 │   30.27 ms │               30.28 ms │     no change │
│ QQuery 85 │  100.40 ms │               99.64 ms │     no change │
│ QQuery 86 │   26.31 ms │               26.83 ms │     no change │
│ QQuery 87 │   65.76 ms │               65.23 ms │     no change │
│ QQuery 88 │   60.97 ms │               60.09 ms │     no change │
│ QQuery 89 │   36.28 ms │               35.94 ms │     no change │
│ QQuery 90 │   17.26 ms │               17.09 ms │     no change │
│ QQuery 91 │   42.51 ms │               42.49 ms │     no change │
│ QQuery 92 │   31.06 ms │               30.33 ms │     no change │
│ QQuery 93 │   50.66 ms │               51.52 ms │     no change │
│ QQuery 94 │   40.40 ms │               40.98 ms │     no change │
│ QQuery 95 │   82.89 ms │               81.56 ms │     no change │
│ QQuery 96 │   24.41 ms │               24.18 ms │     no change │
│ QQuery 97 │   53.07 ms │               52.80 ms │     no change │
│ QQuery 98 │   46.58 ms │               45.79 ms │     no change │
│ QQuery 99 │   67.41 ms │               66.78 ms │     no change │
└───────────┴────────────┴────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                     ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                     │ 9473.77ms │
│ Total Time (hash-join-grouped-rows)   │ 9440.16ms │
│ Average Time (HEAD)                   │   95.69ms │
│ Average Time (hash-join-grouped-rows) │   95.36ms │
│ Queries Faster                        │         1 │
│ Queries Slower                        │         3 │
│ Queries with No Change                │        95 │
│ Queries with Failure                  │         0 │
└───────────────────────────────────────┴───────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and hash-join-grouped-rows
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃                hash-join-grouped-rows ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │           6.04 / 6.58 ±0.88 / 8.33 ms │           5.98 / 6.46 ±0.88 / 8.22 ms │     no change │
│ QQuery 2  │        81.55 / 82.15 ±0.48 / 82.98 ms │        80.99 / 81.49 ±0.38 / 82.11 ms │     no change │
│ QQuery 3  │        30.13 / 30.42 ±0.21 / 30.74 ms │        29.68 / 29.91 ±0.25 / 30.25 ms │     no change │
│ QQuery 4  │     499.97 / 516.58 ±9.79 / 528.88 ms │     498.11 / 507.92 ±7.82 / 518.54 ms │     no change │
│ QQuery 5  │        52.92 / 53.38 ±0.41 / 54.15 ms │        51.59 / 52.02 ±0.32 / 52.45 ms │     no change │
│ QQuery 6  │        37.26 / 37.85 ±0.46 / 38.67 ms │        38.45 / 38.76 ±0.41 / 39.55 ms │     no change │
│ QQuery 7  │        73.56 / 74.80 ±0.70 / 75.59 ms │        73.89 / 74.30 ±0.41 / 74.86 ms │     no change │
│ QQuery 8  │        38.33 / 40.69 ±3.71 / 48.07 ms │        38.06 / 39.45 ±2.08 / 43.57 ms │     no change │
│ QQuery 9  │        54.54 / 56.40 ±2.19 / 60.34 ms │        52.79 / 54.63 ±1.22 / 55.95 ms │     no change │
│ QQuery 10 │        62.12 / 62.31 ±0.21 / 62.68 ms │        61.80 / 62.63 ±0.43 / 63.00 ms │     no change │
│ QQuery 11 │     321.42 / 325.03 ±2.13 / 327.29 ms │     315.99 / 329.09 ±8.95 / 342.20 ms │     no change │
│ QQuery 12 │        30.00 / 30.70 ±0.62 / 31.67 ms │        29.64 / 30.31 ±0.39 / 30.80 ms │     no change │
│ QQuery 13 │     118.25 / 119.71 ±1.23 / 121.70 ms │     115.64 / 117.98 ±2.36 / 122.47 ms │     no change │
│ QQuery 14 │     386.16 / 389.24 ±2.03 / 391.62 ms │     386.65 / 393.60 ±4.17 / 398.73 ms │     no change │
│ QQuery 15 │        62.74 / 63.93 ±0.75 / 64.85 ms │        62.21 / 62.87 ±0.51 / 63.38 ms │     no change │
│ QQuery 16 │           6.58 / 6.70 ±0.09 / 6.83 ms │           6.76 / 6.94 ±0.15 / 7.18 ms │     no change │
│ QQuery 17 │        76.94 / 77.83 ±0.66 / 78.93 ms │        78.15 / 79.84 ±1.69 / 83.09 ms │     no change │
│ QQuery 18 │       97.95 / 99.43 ±1.47 / 102.00 ms │      98.58 / 102.13 ±2.73 / 106.22 ms │     no change │
│ QQuery 19 │        43.22 / 44.45 ±1.48 / 47.35 ms │        42.50 / 43.41 ±0.50 / 43.97 ms │     no change │
│ QQuery 20 │        36.72 / 37.74 ±0.74 / 38.90 ms │        38.90 / 39.11 ±0.15 / 39.33 ms │     no change │
│ QQuery 21 │        17.56 / 17.79 ±0.12 / 17.90 ms │        18.10 / 18.90 ±1.47 / 21.85 ms │  1.06x slower │
│ QQuery 22 │        66.26 / 68.26 ±1.16 / 69.63 ms │        70.96 / 71.56 ±0.51 / 72.45 ms │     no change │
│ QQuery 23 │     323.75 / 332.09 ±6.46 / 343.47 ms │     327.67 / 333.01 ±2.98 / 336.30 ms │     no change │
│ QQuery 24 │     210.53 / 213.65 ±2.53 / 217.82 ms │     212.49 / 214.89 ±2.03 / 217.35 ms │     no change │
│ QQuery 25 │     108.87 / 112.82 ±4.84 / 122.02 ms │     107.87 / 110.95 ±2.70 / 114.16 ms │     no change │
│ QQuery 26 │        47.21 / 48.00 ±0.41 / 48.44 ms │        47.05 / 47.54 ±0.38 / 48.03 ms │     no change │
│ QQuery 27 │           5.65 / 5.75 ±0.06 / 5.85 ms │           5.73 / 5.83 ±0.09 / 5.96 ms │     no change │
│ QQuery 28 │        59.92 / 61.80 ±1.25 / 63.59 ms │        60.15 / 63.84 ±3.75 / 70.09 ms │     no change │
│ QQuery 29 │       92.64 / 97.21 ±4.88 / 106.33 ms │       94.10 / 96.20 ±2.37 / 100.73 ms │     no change │
│ QQuery 30 │        34.65 / 35.32 ±0.74 / 36.71 ms │        34.49 / 34.79 ±0.19 / 35.02 ms │     no change │
│ QQuery 31 │     109.36 / 111.24 ±1.20 / 112.36 ms │     112.01 / 114.88 ±2.65 / 119.52 ms │     no change │
│ QQuery 32 │        20.96 / 21.14 ±0.19 / 21.44 ms │        21.44 / 21.87 ±0.52 / 22.84 ms │     no change │
│ QQuery 33 │        39.08 / 41.74 ±4.07 / 49.85 ms │        38.90 / 39.25 ±0.28 / 39.67 ms │ +1.06x faster │
│ QQuery 34 │           6.22 / 6.31 ±0.08 / 6.44 ms │           6.39 / 6.47 ±0.06 / 6.55 ms │     no change │
│ QQuery 35 │        75.60 / 78.37 ±1.62 / 80.13 ms │        77.60 / 78.42 ±0.59 / 79.28 ms │     no change │
│ QQuery 36 │           6.24 / 6.69 ±0.73 / 8.16 ms │           6.30 / 6.42 ±0.08 / 6.55 ms │     no change │
│ QQuery 37 │           7.65 / 7.87 ±0.18 / 8.11 ms │           7.57 / 8.21 ±0.90 / 9.97 ms │     no change │
│ QQuery 38 │        64.77 / 66.69 ±2.95 / 72.53 ms │        64.09 / 65.21 ±0.87 / 66.22 ms │     no change │
│ QQuery 39 │        77.06 / 78.82 ±1.92 / 81.88 ms │        79.21 / 79.57 ±0.24 / 79.88 ms │     no change │
│ QQuery 40 │        24.59 / 25.03 ±0.56 / 26.10 ms │        25.57 / 26.70 ±1.36 / 29.28 ms │  1.07x slower │
│ QQuery 41 │        13.76 / 13.85 ±0.06 / 13.93 ms │        13.60 / 13.86 ±0.18 / 14.16 ms │     no change │
│ QQuery 42 │        24.43 / 25.78 ±1.28 / 28.07 ms │        24.84 / 25.65 ±0.64 / 26.68 ms │     no change │
│ QQuery 43 │           5.30 / 5.38 ±0.05 / 5.43 ms │           5.15 / 5.22 ±0.07 / 5.34 ms │     no change │
│ QQuery 44 │           6.51 / 6.57 ±0.07 / 6.69 ms │           6.09 / 6.18 ±0.07 / 6.30 ms │ +1.06x faster │
│ QQuery 45 │        46.62 / 48.01 ±1.75 / 51.42 ms │        44.45 / 45.77 ±1.10 / 47.59 ms │     no change │
│ QQuery 46 │        13.35 / 14.17 ±0.65 / 14.98 ms │        12.71 / 13.19 ±0.43 / 13.99 ms │ +1.07x faster │
│ QQuery 47 │     259.41 / 266.09 ±5.77 / 274.78 ms │     253.39 / 261.71 ±5.88 / 271.26 ms │     no change │
│ QQuery 48 │        93.90 / 94.45 ±0.44 / 95.04 ms │        93.95 / 95.55 ±1.21 / 97.40 ms │     no change │
│ QQuery 49 │        72.95 / 74.40 ±1.25 / 76.33 ms │        73.12 / 74.25 ±0.89 / 75.83 ms │     no change │
│ QQuery 50 │        58.60 / 59.19 ±0.60 / 60.22 ms │        58.69 / 60.97 ±3.21 / 67.34 ms │     no change │
│ QQuery 51 │       97.21 / 99.80 ±3.56 / 106.34 ms │      98.53 / 100.89 ±1.98 / 104.41 ms │     no change │
│ QQuery 52 │        25.32 / 25.81 ±0.54 / 26.50 ms │        25.36 / 25.93 ±0.41 / 26.46 ms │     no change │
│ QQuery 53 │        30.70 / 30.80 ±0.07 / 30.89 ms │        30.53 / 30.87 ±0.23 / 31.16 ms │     no change │
│ QQuery 54 │        55.10 / 56.00 ±0.91 / 57.74 ms │        56.11 / 58.06 ±3.30 / 64.64 ms │     no change │
│ QQuery 55 │        24.78 / 25.24 ±0.29 / 25.67 ms │        24.75 / 25.66 ±1.09 / 27.76 ms │     no change │
│ QQuery 56 │        39.87 / 40.64 ±0.51 / 41.32 ms │        39.80 / 40.22 ±0.39 / 40.77 ms │     no change │
│ QQuery 57 │     179.77 / 184.01 ±3.66 / 190.74 ms │     178.81 / 180.44 ±1.61 / 183.07 ms │     no change │
│ QQuery 58 │     113.57 / 115.17 ±2.29 / 119.69 ms │     113.11 / 113.87 ±0.50 / 114.60 ms │     no change │
│ QQuery 59 │     118.82 / 120.75 ±2.06 / 124.33 ms │     118.60 / 119.63 ±1.42 / 122.42 ms │     no change │
│ QQuery 60 │        39.70 / 40.48 ±0.78 / 41.92 ms │        39.53 / 40.09 ±0.49 / 40.72 ms │     no change │
│ QQuery 61 │           9.53 / 9.61 ±0.07 / 9.70 ms │           9.63 / 9.76 ±0.13 / 9.99 ms │     no change │
│ QQuery 62 │        44.75 / 45.41 ±0.54 / 46.35 ms │        45.91 / 47.39 ±2.53 / 52.43 ms │     no change │
│ QQuery 63 │        30.53 / 30.78 ±0.14 / 30.91 ms │        30.31 / 31.03 ±0.51 / 31.69 ms │     no change │
│ QQuery 64 │     360.19 / 368.22 ±9.22 / 385.66 ms │     373.00 / 378.43 ±4.89 / 385.96 ms │     no change │
│ QQuery 65 │        72.37 / 74.62 ±1.70 / 77.31 ms │        72.85 / 74.37 ±1.09 / 75.48 ms │     no change │
│ QQuery 66 │        75.70 / 79.91 ±6.13 / 91.96 ms │        77.04 / 78.04 ±0.66 / 79.09 ms │     no change │
│ QQuery 67 │     242.10 / 253.94 ±7.58 / 266.04 ms │     243.80 / 250.68 ±5.36 / 256.21 ms │     no change │
│ QQuery 68 │           7.08 / 7.24 ±0.14 / 7.44 ms │           6.95 / 7.02 ±0.06 / 7.11 ms │     no change │
│ QQuery 69 │        56.99 / 58.42 ±0.92 / 59.83 ms │        57.15 / 58.04 ±0.61 / 58.91 ms │     no change │
│ QQuery 70 │     105.67 / 111.56 ±5.86 / 121.49 ms │     108.75 / 109.31 ±0.55 / 110.33 ms │     no change │
│ QQuery 71 │        35.56 / 36.37 ±0.76 / 37.32 ms │        36.00 / 37.93 ±2.35 / 42.54 ms │     no change │
│ QQuery 72 │ 1820.40 / 1912.95 ±78.30 / 2044.06 ms │ 1765.25 / 1851.67 ±48.30 / 1902.75 ms │     no change │
│ QQuery 73 │           5.57 / 5.71 ±0.10 / 5.88 ms │           5.93 / 6.02 ±0.09 / 6.17 ms │  1.06x slower │
│ QQuery 74 │     224.56 / 231.49 ±5.14 / 239.94 ms │     224.12 / 226.56 ±2.60 / 231.51 ms │     no change │
│ QQuery 75 │     141.47 / 143.65 ±1.54 / 146.25 ms │     143.77 / 144.95 ±0.94 / 146.65 ms │     no change │
│ QQuery 76 │        35.80 / 38.75 ±4.31 / 47.33 ms │        35.41 / 35.89 ±0.34 / 36.47 ms │ +1.08x faster │
│ QQuery 77 │        61.48 / 61.97 ±0.41 / 62.68 ms │        62.07 / 62.73 ±0.45 / 63.36 ms │     no change │
│ QQuery 78 │    285.27 / 295.53 ±11.88 / 317.07 ms │    286.03 / 307.70 ±16.45 / 329.59 ms │     no change │
│ QQuery 79 │        69.47 / 75.37 ±6.96 / 89.05 ms │        70.56 / 71.45 ±0.51 / 71.94 ms │ +1.05x faster │
│ QQuery 80 │      99.88 / 101.22 ±1.58 / 104.17 ms │      99.17 / 102.68 ±3.65 / 109.01 ms │     no change │
│ QQuery 81 │        27.15 / 28.34 ±1.30 / 30.85 ms │        27.79 / 28.51 ±1.21 / 30.92 ms │     no change │
│ QQuery 82 │        17.26 / 20.02 ±5.08 / 30.18 ms │        17.48 / 17.70 ±0.20 / 17.98 ms │ +1.13x faster │
│ QQuery 83 │        34.12 / 34.51 ±0.32 / 34.98 ms │        34.50 / 34.94 ±0.42 / 35.65 ms │     no change │
│ QQuery 84 │        30.27 / 30.89 ±0.62 / 32.01 ms │        30.28 / 30.58 ±0.19 / 30.83 ms │     no change │
│ QQuery 85 │     100.40 / 102.38 ±1.60 / 105.19 ms │      99.64 / 104.19 ±6.30 / 116.64 ms │     no change │
│ QQuery 86 │        26.31 / 26.71 ±0.32 / 27.07 ms │        26.83 / 27.76 ±0.75 / 29.04 ms │     no change │
│ QQuery 87 │        65.76 / 68.83 ±3.87 / 76.46 ms │        65.23 / 66.72 ±1.14 / 68.38 ms │     no change │
│ QQuery 88 │        60.97 / 61.83 ±0.69 / 63.02 ms │        60.09 / 61.17 ±0.90 / 62.29 ms │     no change │
│ QQuery 89 │        36.28 / 36.63 ±0.39 / 37.32 ms │        35.94 / 36.73 ±0.55 / 37.42 ms │     no change │
│ QQuery 90 │        17.26 / 17.38 ±0.10 / 17.50 ms │        17.09 / 17.81 ±1.08 / 19.94 ms │     no change │
│ QQuery 91 │        42.51 / 44.28 ±2.88 / 50.00 ms │        42.49 / 42.68 ±0.20 / 43.06 ms │     no change │
│ QQuery 92 │        31.06 / 31.42 ±0.34 / 32.04 ms │        30.33 / 31.09 ±0.55 / 31.89 ms │     no change │
│ QQuery 93 │        50.66 / 51.49 ±0.61 / 52.20 ms │        51.52 / 53.40 ±1.54 / 56.04 ms │     no change │
│ QQuery 94 │        40.40 / 41.37 ±1.17 / 43.51 ms │        40.98 / 42.48 ±1.83 / 46.08 ms │     no change │
│ QQuery 95 │        82.89 / 85.18 ±2.40 / 89.79 ms │        81.56 / 83.23 ±1.55 / 85.74 ms │     no change │
│ QQuery 96 │        24.41 / 24.67 ±0.22 / 25.04 ms │        24.18 / 24.42 ±0.15 / 24.63 ms │     no change │
│ QQuery 97 │        53.07 / 54.09 ±1.27 / 56.32 ms │        52.80 / 53.91 ±1.24 / 55.65 ms │     no change │
│ QQuery 98 │        46.58 / 47.19 ±0.59 / 48.28 ms │        45.79 / 49.56 ±4.83 / 59.04 ms │  1.05x slower │
│ QQuery 99 │        67.41 / 69.41 ±3.04 / 75.44 ms │        66.78 / 68.02 ±0.73 / 68.66 ms │     no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                     ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                     │ 9758.44ms │
│ Total Time (hash-join-grouped-rows)   │ 9701.93ms │
│ Average Time (HEAD)                   │   98.57ms │
│ Average Time (hash-join-grouped-rows) │   98.00ms │
│ Queries Faster                        │         6 │
│ Queries Slower                        │         4 │
│ Queries with No Change                │        89 │
│ Queries with Failure                  │         0 │
└───────────────────────────────────────┴───────────┘

Resource Usage

tpcds — base (merge-base)

Metric Value
Wall time 50.0s
Peak memory 2.0 GiB
Avg memory 1.5 GiB
CPU user 206.9s
CPU sys 6.3s
Peak spill 0 B

tpcds — branch

Metric Value
Wall time 50.0s
Peak memory 2.2 GiB
Avg memory 1.5 GiB
CPU user 202.1s
CPU sys 6.2s
Peak spill 0 B

File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing hash-join-grouped-rows (104d556) to 477c7b9 (merge-base) diff

Run configuration
run benchmark clickbench_partitioned
CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and hash-join-grouped-rows
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ hash-join-grouped-rows ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │    1.41 ms │                1.41 ms │     no change │
│ QQuery 1  │   12.42 ms │               12.38 ms │     no change │
│ QQuery 2  │   35.56 ms │               36.23 ms │     no change │
│ QQuery 3  │   32.93 ms │               33.44 ms │     no change │
│ QQuery 4  │  279.32 ms │              238.73 ms │ +1.17x faster │
│ QQuery 5  │  314.61 ms │              275.66 ms │ +1.14x faster │
│ QQuery 6  │    1.46 ms │                1.20 ms │ +1.22x faster │
│ QQuery 7  │   14.53 ms │               13.63 ms │ +1.07x faster │
│ QQuery 8  │  330.15 ms │              326.99 ms │     no change │
│ QQuery 9  │  472.38 ms │              461.16 ms │     no change │
│ QQuery 10 │   64.30 ms │               64.57 ms │     no change │
│ QQuery 11 │   75.30 ms │               76.06 ms │     no change │
│ QQuery 12 │  259.25 ms │              261.38 ms │     no change │
│ QQuery 13 │  370.37 ms │              373.74 ms │     no change │
│ QQuery 14 │  275.86 ms │              277.11 ms │     no change │
│ QQuery 15 │  282.84 ms │              283.35 ms │     no change │
│ QQuery 16 │  683.58 ms │              701.54 ms │     no change │
│ QQuery 17 │  634.13 ms │              627.02 ms │     no change │
│ QQuery 18 │ 1189.71 ms │             1211.85 ms │     no change │
│ QQuery 19 │   27.48 ms │               27.81 ms │     no change │
│ QQuery 20 │  518.26 ms │              525.48 ms │     no change │
│ QQuery 21 │  511.44 ms │              540.37 ms │  1.06x slower │
│ QQuery 22 │  979.70 ms │              981.70 ms │     no change │
│ QQuery 23 │ 3054.81 ms │             3044.48 ms │     no change │
│ QQuery 24 │   40.71 ms │               41.52 ms │     no change │
│ QQuery 25 │  106.18 ms │              106.95 ms │     no change │
│ QQuery 26 │   41.43 ms │               41.81 ms │     no change │
│ QQuery 27 │  505.22 ms │              513.20 ms │     no change │
│ QQuery 28 │ 2904.48 ms │             2891.74 ms │     no change │
│ QQuery 29 │   37.68 ms │               37.95 ms │     no change │
│ QQuery 30 │  315.20 ms │              334.48 ms │  1.06x slower │
│ QQuery 31 │  331.09 ms │              304.97 ms │ +1.09x faster │
│ QQuery 32 │  946.04 ms │              936.79 ms │     no change │
│ QQuery 33 │ 1455.94 ms │             1418.99 ms │     no change │
│ QQuery 34 │ 1489.68 ms │             1526.68 ms │     no change │
│ QQuery 35 │  291.93 ms │              281.08 ms │     no change │
│ QQuery 36 │   65.86 ms │               65.94 ms │     no change │
│ QQuery 37 │   34.29 ms │               33.78 ms │     no change │
│ QQuery 38 │   38.49 ms │               39.15 ms │     no change │
│ QQuery 39 │  148.06 ms │              143.63 ms │     no change │
│ QQuery 40 │   13.50 ms │               13.44 ms │     no change │
│ QQuery 41 │   12.59 ms │               12.71 ms │     no change │
│ QQuery 42 │   11.87 ms │               11.86 ms │     no change │
└───────────┴────────────┴────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                     ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                     │ 19212.04ms │
│ Total Time (hash-join-grouped-rows)   │ 19153.93ms │
│ Average Time (HEAD)                   │   446.79ms │
│ Average Time (hash-join-grouped-rows) │   445.44ms │
│ Queries Faster                        │          5 │
│ Queries Slower                        │          2 │
│ Queries with No Change                │         36 │
│ Queries with Failure                  │          0 │
└───────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and hash-join-grouped-rows
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                   HEAD ┃                 hash-join-grouped-rows ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │            1.41 / 2.07 ±1.14 / 4.35 ms │            1.41 / 2.07 ±1.15 / 4.38 ms │     no change │
│ QQuery 1  │         12.42 / 12.59 ±0.11 / 12.74 ms │         12.38 / 13.09 ±0.53 / 13.91 ms │     no change │
│ QQuery 2  │         35.56 / 36.00 ±0.39 / 36.51 ms │         36.23 / 37.16 ±1.19 / 39.46 ms │     no change │
│ QQuery 3  │         32.93 / 33.57 ±0.60 / 34.71 ms │         33.44 / 33.74 ±0.25 / 34.16 ms │     no change │
│ QQuery 4  │      279.32 / 282.30 ±2.90 / 287.23 ms │     238.73 / 265.86 ±17.77 / 288.73 ms │ +1.06x faster │
│ QQuery 5  │      314.61 / 320.81 ±5.19 / 329.86 ms │      275.66 / 279.29 ±3.10 / 285.05 ms │ +1.15x faster │
│ QQuery 6  │            1.46 / 1.62 ±0.20 / 2.02 ms │            1.20 / 1.37 ±0.25 / 1.85 ms │ +1.19x faster │
│ QQuery 7  │         14.53 / 14.73 ±0.15 / 14.92 ms │         13.63 / 13.70 ±0.06 / 13.80 ms │ +1.08x faster │
│ QQuery 8  │     330.15 / 356.00 ±20.81 / 383.90 ms │      326.99 / 332.05 ±3.27 / 337.17 ms │ +1.07x faster │
│ QQuery 9  │      472.38 / 479.60 ±6.58 / 489.33 ms │      461.16 / 467.71 ±4.71 / 474.45 ms │     no change │
│ QQuery 10 │         64.30 / 65.14 ±0.86 / 66.77 ms │         64.57 / 65.61 ±1.48 / 68.45 ms │     no change │
│ QQuery 11 │         75.30 / 76.46 ±1.12 / 78.57 ms │         76.06 / 76.79 ±0.45 / 77.43 ms │     no change │
│ QQuery 12 │      259.25 / 267.55 ±6.12 / 277.13 ms │      261.38 / 268.46 ±4.49 / 274.54 ms │     no change │
│ QQuery 13 │      370.37 / 378.48 ±6.47 / 388.44 ms │      373.74 / 390.61 ±8.81 / 398.40 ms │     no change │
│ QQuery 14 │      275.86 / 281.56 ±4.49 / 288.02 ms │      277.11 / 284.71 ±5.59 / 293.43 ms │     no change │
│ QQuery 15 │      282.84 / 288.01 ±4.43 / 295.56 ms │     283.35 / 308.07 ±29.44 / 349.78 ms │  1.07x slower │
│ QQuery 16 │     683.58 / 709.59 ±15.22 / 727.48 ms │      701.54 / 711.72 ±9.51 / 727.70 ms │     no change │
│ QQuery 17 │     634.13 / 670.14 ±32.51 / 712.88 ms │     627.02 / 662.05 ±39.38 / 736.88 ms │     no change │
│ QQuery 18 │  1189.71 / 1232.43 ±27.19 / 1273.51 ms │  1211.85 / 1234.63 ±15.31 / 1259.14 ms │     no change │
│ QQuery 19 │        27.48 / 35.93 ±10.49 / 53.09 ms │         27.81 / 28.33 ±0.58 / 29.08 ms │ +1.27x faster │
│ QQuery 20 │     518.26 / 532.57 ±14.81 / 558.83 ms │     525.48 / 535.17 ±13.58 / 560.84 ms │     no change │
│ QQuery 21 │     511.44 / 532.55 ±11.71 / 546.59 ms │      540.37 / 546.88 ±6.16 / 556.40 ms │     no change │
│ QQuery 22 │   979.70 / 1030.72 ±42.88 / 1094.68 ms │   981.70 / 1017.03 ±33.83 / 1068.84 ms │     no change │
│ QQuery 23 │  3054.81 / 3161.99 ±84.29 / 3257.76 ms │ 3044.48 / 3190.03 ±106.40 / 3347.30 ms │     no change │
│ QQuery 24 │         40.71 / 43.88 ±5.99 / 55.85 ms │         41.52 / 45.68 ±4.36 / 51.96 ms │     no change │
│ QQuery 25 │      106.18 / 112.79 ±9.23 / 130.82 ms │     106.95 / 115.06 ±11.03 / 135.53 ms │     no change │
│ QQuery 26 │         41.43 / 41.88 ±0.44 / 42.57 ms │         41.81 / 44.15 ±4.16 / 52.46 ms │  1.05x slower │
│ QQuery 27 │     505.22 / 519.80 ±10.03 / 535.20 ms │      513.20 / 524.68 ±9.02 / 535.65 ms │     no change │
│ QQuery 28 │  2904.48 / 2968.64 ±61.90 / 3052.63 ms │  2891.74 / 2960.13 ±58.46 / 3043.28 ms │     no change │
│ QQuery 29 │         37.68 / 39.22 ±2.39 / 43.99 ms │        37.95 / 47.27 ±10.85 / 61.20 ms │  1.21x slower │
│ QQuery 30 │     315.20 / 349.62 ±18.82 / 366.45 ms │     334.48 / 351.58 ±11.69 / 369.81 ms │     no change │
│ QQuery 31 │     331.09 / 345.76 ±11.07 / 359.43 ms │     304.97 / 317.26 ±11.15 / 335.77 ms │ +1.09x faster │
│ QQuery 32 │   946.04 / 1007.06 ±54.38 / 1082.02 ms │  936.79 / 1046.31 ±100.43 / 1203.19 ms │     no change │
│ QQuery 33 │  1455.94 / 1475.56 ±19.64 / 1499.57 ms │  1418.99 / 1476.35 ±46.36 / 1518.61 ms │     no change │
│ QQuery 34 │ 1489.68 / 1647.42 ±105.02 / 1748.45 ms │  1526.68 / 1637.74 ±91.50 / 1748.68 ms │     no change │
│ QQuery 35 │     291.93 / 324.61 ±26.80 / 354.35 ms │    281.08 / 364.67 ±143.28 / 650.59 ms │  1.12x slower │
│ QQuery 36 │         65.86 / 71.75 ±5.12 / 78.76 ms │       65.94 / 83.78 ±24.10 / 131.45 ms │  1.17x slower │
│ QQuery 37 │         34.29 / 42.16 ±7.85 / 56.63 ms │         33.78 / 34.08 ±0.28 / 34.57 ms │ +1.24x faster │
│ QQuery 38 │         38.49 / 41.60 ±3.43 / 47.79 ms │         39.15 / 43.06 ±4.72 / 52.32 ms │     no change │
│ QQuery 39 │      148.06 / 157.70 ±9.46 / 172.41 ms │      143.63 / 149.04 ±6.72 / 162.08 ms │ +1.06x faster │
│ QQuery 40 │         13.50 / 14.70 ±1.78 / 18.23 ms │         13.44 / 13.82 ±0.32 / 14.26 ms │ +1.06x faster │
│ QQuery 41 │         12.59 / 13.60 ±1.80 / 17.19 ms │         12.71 / 14.21 ±2.64 / 19.47 ms │     no change │
│ QQuery 42 │         11.87 / 12.82 ±1.13 / 15.02 ms │         11.86 / 12.86 ±1.64 / 16.11 ms │     no change │
└───────────┴────────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                     ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                     │ 20032.96ms │
│ Total Time (hash-join-grouped-rows)   │ 20047.88ms │
│ Average Time (HEAD)                   │   465.88ms │
│ Average Time (hash-join-grouped-rows) │   466.23ms │
│ Queries Faster                        │         10 │
│ Queries Slower                        │          5 │
│ Queries with No Change                │         28 │
│ Queries with Failure                  │          0 │
└───────────────────────────────────────┴────────────┘

Resource Usage

clickbench_partitioned — base (merge-base)

Metric Value
Wall time 105.0s
Peak memory 12.1 GiB
Avg memory 4.1 GiB
CPU user 1021.6s
CPU sys 76.4s
Peak spill 0 B

clickbench_partitioned — branch

Metric Value
Wall time 105.0s
Peak memory 11.9 GiB
Avg memory 4.8 GiB
CPU user 1017.0s
CPU sys 77.2s
Peak spill 0 B

File an issue against this benchmark runner

@andygrove

Copy link
Copy Markdown
Member Author

Thanks for running these. I looked into the queries flagged as slower:

  • ClickBench has no joins, so the 5% to 27% swings both ways there are noise.
  • TPC-H Q14 and Q19 are flagged in only one of the two tables.
  • Of the TPC-DS queries flagged slower (Q20, Q21, Q22, Q40, Q73), only Q40 groups any map (1 of its 4). The others never reach the grouping code. For them the PR only adds a counter on insert and an early return before grouping.

I ran an A/B of the same two commits (477c7b9 and 104d556) with datafusion-cli on an AMD 7950X3D. It took 20 warm runs per query, over three rounds alternating between builds. Every flagged query is within 1 ms of main:

Query Maps grouped main (min / median) this PR (min / median)
TPC-DS Q20 0 11 / 12 ms 11 / 12–13 ms
TPC-DS Q21 0 14–15 / 15 ms 14–15 / 15 ms
TPC-DS Q22 0 304–311 / 314–316 ms 307–310 / 312–314 ms
TPC-DS Q40 1 of 4 95–96 / 97–100 ms 95 / 98–99 ms
TPC-DS Q73 0 5 / 5 ms 5 / 5 ms
TPC-H Q14 0 27–28 / 28–29 ms 27 / 29 ms
TPC-H Q19 0 37 / 38–39 ms 36–37 / 38–39 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation physical-plan Changes to the physical-plan crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants